Memory module having signal lines configured for sequential arrival of signals at a plurality of memory devices

ABSTRACT

A memory module includes a substrate, a plurality of signal lines, a clock line and a plurality of memory devices. The plurality of signal lines including first and second signal lines routed alongside one another where, for each of the first and second signal lines, a respective signal, starting at a corresponding first edge finger, traverses in sequence, a respective first segment of a respective signal line, a respective turn portion of the respective signal line, and a respective second segment of the respective signal line. The clock line is to provide a clock signal that traverses in sequence, a second edge finger, the first segment of the clock line, the turn portion of the clock line, and the second segment of the clock line. The respective signals traverse and the clock signal line arrive at the plurality of memory devices in sequence.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.12/975,313, filed Dec. 21, 2010, which is a continuation of U.S.application Ser. No. 12/426,083, filed Apr. 17, 2009, now U.S. Pat. No.7,870,322, which is a continuation of U.S. patent application Ser. No.11/459,858, filed Jul. 25, 2006, now U.S. Pat. No. 7,523,244, which is acontinuation of U.S. patent application Ser. No. 09/839,768, filed Apr.19, 2001, now U.S. Pat. No. 7,085,872, which is a continuation of U.S.patent application Ser. No. 09/507,303, filed Feb. 18, 2000, now U.S.Pat. No. 6,266,730, which is a continuation of U.S. patent applicationSer. No. 08/938,084, filed Sep. 26, 1997, which is now U.S. Pat. No.6,067,594, which applications are incorporated by reference herein intheir entirety.

BACKGROUND

As computer processors increase in speed they require increasedinformation bandwidth from other subsystems supporting the processor. Anexample is the large amount of bandwidth needed by video and 3D imageprocessing from a computer memory subsystem. Another example is a mainmemory subsystem. One or more high frequency buses are typicallyemployed to provide the bandwidth required. The higher the frequency ofoperation of the bus, the greater the requirement that the signals onthe bus have high-fidelity and equal propagation times to the devicesmaking up the subsystem. High-fidelity signals are signals having littleor no ringing and controlled and steady rising and falling edge rates.Many obstacles are encountered in assuring the uniform arrival times ofhigh-fidelity signals to devices on the bus. One such obstacle is arequirement that a subsystem be modular, meaning that portions of asubsystem may be added and possibly removed. The modularity requirementimplies that devices that are part of the modular subsystem be mountedon a separate substrate or module which couples to another board, themotherboard. It also implies the use of connectors if both addition andremoval is required. Other obstacles are the number of layers of themotherboard on which routing of the bus is allowed and whether the busis routed in a straight line or routed with turns. Too few layers on amotherboard, or a module, and turns of the lines may not permit theconstruction of the bus lines in a way necessary to achieve uniformarrival times of high-fidelity signals to devices on the bus.

Modular subsystems in computers have numerous advantages, some of whichare field upgradeability, replacement of a failing device, flexibilityof initial configuration, and increased device density. Currently, socalled SIMMs (single in-line memory module) and DIMMs (dual in-linememory modules) are examples of computer memory systems employing suchmodules. Because of these advantages and the desirability of having highperformance modular memory subsystems, it is especially important tohave buses with uniform arrival times to devices in applications wheremodules are employed.

One form of module technology, using buses, is oriented to a gridtopology having three groups of lines as shown in FIG. 1. In the firstgroup 120, all of the lines connect to all devices on all modules 140a-c. In the second group 110, the group is partitioned into a number ofsubgroups, 112, 114, 116, and 118, which connect to a correspondingdevice in each module. For example, in FIG. 1, a portion, say 1, 4, or 8lines are routed to a similarly situated device in each module. In thethird group 130, the lines are typically radially connected to themodules and all devices in a particular module connect to the dedicatedradial line or lines. For a memory module, the first group 120 isrepresentative of address bus and clock lines, the second group 110 ofthe data bus or buses and the third group 130 of the control lines, suchas RAS and CAS. Corresponding to each group is a representativetransmission line having a certain set of characteristics, such aspropagation delay and loaded or unloaded characteristic impedance, whichare different for each group. This leads to difficulty in matching thearrival of signals of each group at the devices on the modules andlimits the performance that can be obtained from such a topology due towaiting on the delays of the slowest group of lines, which waiting alsoincludes the settling of the lines when not properly terminated.

A circuit model of a tapped line, typical of the second group of lines,is shown in FIG. 3. As shown, in this topology, each line in a group istypically connected to a module by means of a stub 360 which acts as tapoff of the line as shown in FIG. 2 and FIG. 3. A stub is defined as alength of line tapped from a transmission line and having a round tripdelay which is greater than the rise time (or fall time) of the signal.Since the stub 360 (160 in FIG. 2) typically has a different impedancethan the line being tapped, it is often necessary to insert a resistor320, as shown in FIG. 3, in series with the stub to mitigate the effectof reflections at the connection point of the stub to the line. If theline impedance is about 50 ohms and the impedance of the stub is about75 ohms, a resistor of approximately 20-25 ohms is typically chosenusually by trial and error for the best results under certainconditions. This resistor has the possibly undesirable effect ofattenuating the voltage swing of the signal as the signal passes throughthe resistor, requiring a driver on the stub to have a proportionatelylarger voltage swing. Another undesirable effect is the RC delay due tothe added series resistor and the device capacitance. The resistors andstubs also lead to low-fidelity signals at the devices. Also, as shownin FIG. 3, the line is terminated by resistors 350 at both ends tominimize reflections from the ends of the line. This requires that thedrivers on the line supply twice as much steady state current ascompared to a line terminated at only one end.

As mentioned above, the need to incorporate memory modules into thedesign of the modular system may also imply the use of connectors. Ingeneral, connectors have undesirable characteristics for operating athigh frequency, such as inductance, capacitance, or crosstalk whichintroduces noise from one line into another line. Failure to take theconnector characteristics into account leads to non-uniform arrivalcharacteristics and low-fidelity signals when crossing a connectorboundary resulting in lower performance (due to longer settling times,reduced noise margin or different signal propagation speed) from themodular system using the lines.

The physical shape, size and construction of the memory module isimportant to consider as well. The physical nature of the memory modulemay force the IC devices mounted on the module to be arranged in aless-than-optimal topology for the high frequency transmission linelayout. High frequency signaling typically requires that electricalpaths be controlled; signal delays need to be minimized or matched andimpedance needs to be tightly controlled for high frequency operation,where high frequency means frequencies in the range of 200 megahertz toat least 1,000 megaHertz.

For the foregoing reasons, there is a need for a bus connecting to aplurality of devices which has uniform arrival times of high-fidelitysignals to the devices on the bus, even when modules and connectors areemployed to build a computer subsystem in which the bus is used anddespite the physical size, shape and construction of the module and thenumber of devices mounted on it.

SUMMARY OF THE INVENTION

The present invention is directed to a high frequency bus system whichinsures uniform arrival times of high-fidelity signals to the devices onthe high frequency bus, despite the use of the bus on modules andconnectors. A high frequency bus system includes a first bus segmenthaving one or more devices connected between a first and a second end.The first bus segment has at least a pair of transmission lines forpropagating high frequency signals and the devices are coupled to thepair of transmission lines. The high frequency bus system also includesa second bus segment which has no devices connected to it. The secondbus segment also has at least a pair of transmission lines forpropagating high frequency signals. The first end of the first segmentand second end of the second segment are coupled in series to form achain of segments and when two signals are introduced to the first endof the second bus segment at the substantially the same time, theyarrive at each device connected to the first bus segment atsubstantially the same time. Also, when two signals originate at adevice connected to the first bus segment at substantially the sametime, they arrive at the first end of the second bus segment atsubstantially the same time. Uniform arrival times hold despite the useof connectors to couple the segments together, despite the segmentsbeing located on modules, without the need for stubs, despite thepresence of routing turns in the segments and despite the type ofinformation, such as address, data or control, carried by the signals.

In a preferred embodiment, the bus runs along a motherboard (secondsegment), onto one end of a memory module, then along the memory module(first segment), and exits the other end of the memory module alonganother motherboard segment to a next memory module. By running the busthrough the module, stubs between a motherboard bus and each device areavoided, eliminating the need for resistors to compensate forreflections caused by the stubs. Preferably, each bus goes to alldevices, using a control packet to select particular devices as needed.

In a preferred embodiment, the uniform arrival time is also insured by anumber of routing and impedance modifying techniques. In particular, thetotal length traveled by different busses is equalized by having shorterhorizontal length bus lines connected with corresponding longer verticallengths joined by a right angle. In another aspect, parallel bus linesare used to equalize impedance for internally routed lines, since theyhave dielectric material on both sides, compared to a surface line withdielectric on only one side.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows the routing topology for a prior art modular memory system.

FIG. 2 shows a prior art memory module of the type employed in FIG. 1.

FIG. 3 shows a circuit model of a transmission line of a prior artmodule used in FIG. 1, including the terminators and the stubs.

FIG. 4 gives a perspective view of an embodiment of the presentinvention.

FIG. 5 shows a representative module of an embodiment of the presentinvention.

FIG. 6 gives a pictorial view of the routing of the transmission linesfor an embodiment of the present invention.

FIG. 7 gives a pictorial view of the routing of the transmission lineson the motherboard for an embodiment of the present invention.

FIG. 8 gives a simplified view of the routing of a data transmissionline for an embodiment of the present invention.

FIG. 9 shows a simplified view of the routing of a clock transmissionline for an embodiment of the present invention.

FIG. 10 shows a simplified view of an alternative routing of a datatransmission line for an embodiment of the present invention.

FIG. 11 shows another alternative routing for a data transmission lineof an embodiment of the present invention.

FIG. 12 depicts a circuit model of a data transmission line near theconnection point between the motherboard and the module for anembodiment of the present invention.

FIG. 13A depicts a circuit model of a data transmission line on themodule for an embodiment of the present invention.

FIG. 13B depicts a circuit model of a device load on the module for anembodiment of the present invention.

FIG. 14A shows the routing of the transmission lines on the module nearthe right angle turn of the lines for an embodiment of the presentinvention.

FIG. 14B shows a magnified view of the routing of the transmission lineson the module near the right angle turn for an embodiment of the presentinvention.

FIG. 15A shows a perspective view from one side of the module of therouting of the transmission lines on the module near the right angleturn of the lines for an embodiment of the present invention.

FIG. 15B shows a perspective view from the opposite side shown in FIG.15A of the module of the routing of the transmission lines on the modulenear the right angle turn of the lines for an embodiment of the presentinvention.

FIGS. 16A-16E show cross-section views of a module in variousembodiments of the present invention.

FIG. 17 shows a microstrip with the important parameters for computingthe impedance and velocity of the signals on the transmission line of amodule for an embodiment of the present invention.

FIG. 18 shows a stripline with the important parameters for computingthe impedance and velocity of the signals on the transmission line foran embodiment of the present invention.

FIG. 19 shows a stripline with two internal transmission lines and theimportant parameters for computing the impedance and velocity of thesignals on the transmission line for an embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 4 shows a perspective view of an embodiment of the presentinvention. Motherboard 410 acts as the substrate to which device modules420 are coupled. A device 400 on the motherboard acts as a master whichcontrols the devices on the modules. A high frequency bus 450 is routedfrom master 400 on the motherboard, through each module and finally to aterminator 440 on the motherboard. Each module 420 has a marking or key430 to assure that each module is coupled with the same orientation tothe motherboard. A device module 420 may or may not have devices mountedon it. A module with no devices still has the bus routed through it andis termed a continuity module. Continuity modules are used to preservethe continuity of the bus when modules must be removed.

A more detailed view of module 420 is shown in FIG. 5 as module 520.Module 520 has devices 510 mounted one side or both sides of the module.The module also includes a set of edge fingers 540, 550 for coupling tothe motherboard and bus transmission lines 530 which enter the module onthe edge fingers 540 of one side, make a right angle turn at afeedthrough hole 580, traverse along the length of the module connectingto the devices on either side of the module, make another right angleturn and exit on the opposite side edge fingers 550. Edge fingers have acertain distance between their center lines (referred to as pitch) theconnection points on the devices have a certain pitch. The pitch puts anupper limit on the width of a line connecting to the fingers 540, 550 orthe devices. For example, if the distance between edge fingers (measuredfrom center line to center line) is about 30 mils (thousands of aninch), then the lines 530 which connect to the edge fingers must be lessthan about 22 mils in width, allowing for an 8 mil separation betweenlines. Also depicted are edge fingers 595 connecting to a referenceplane within the module by means of lines 585. This preserves therelationship of grounds between signals as the signals pass through theconnector joining the module to the motherboard.

Notches 560 on the side having the edge fingers are used as a key toassure the proper orientation and electrical and parametriccompatibility of the module when coupled to the motherboard. Notches 570may be used by a clamping or retaining device to hold the device modulein place on the motherboard.

FIG. 6 gives a pictorial view of the routing of a data transmission lineand a clock transmission line for an embodiment of the presentinvention. Data line or lines 650 are routed on the motherboard frommaster or controller 600 to device module 620 a. The lines then traversethe length of the module substantially parallel to the long side of themodule and exit at the opposite end from the entry onto the module.Along the length of the line on the module each device 655, situated oneither side of the module, is connected to the line. The line is routedon the motherboard again until it enters the next module 620 b at oneend, again traversing the length of the module substantially parallel tothe long side of the module and exiting at the opposite end. At the exitfrom module 620 c the line then runs on the motherboard to a terminationdevice 640. Clock line 660 starts at a clock source 680 and entersmodule 620 c. It traverses the module 620 c connecting to each deviceand exits from 620 c onto the motherboard. The clock line 660 continuessimilarly until it reaches controller 600. Near or within controller 600the clock line 660 is looped back to join to clock line 670 whichtravels back through the modules in a similar fashion to the routing forthe data line. Upon exiting the module 620 c the clock line runs on themotherboard to a termination device 645. It is important for the properoperation of the devices that the clock lines 660 and 670 be routed in afashion similar to the data line 650 and that no relative changes indirection of propagation occur between the two lines. This means that ifa signal is propagating on the data line 650 of module 620 a away fromthe dot (the orientation dot shown in the lower left of each module)then the signal on the clock line 670 propagates in the same directionand the signal on clock line 660 propagates in the opposite direction.This relationship is preserved between the data line and the clock lineson the motherboard and on the other modules.

FIG. 8 shows a more simple version of the routing of a data line in anembodiment of the present invention. Sections 800 are bus segments ofthe data line on the motherboard. Sections 810 a-c are bus segments ofthe data line on the module with the module orientation key 830 alsoshown for the module. Couplings between the motherboard bus segments 800and the module bus segments 850 are also shown. These couplings can beeither permanent connections such as connections by means of pins on themodule soldered into through-holes on the mother board, or removableconnections such as in the case of a connector. The terminator for thebus segment 840 is shown at the end of the line. Also depicted are rightangle turns 860 at both ends of the bus segment on the module. Theseright angle turns are necessary to assure substantially equal lengthsfor each data line routed according to FIG. 8, when lines connect to themodule from a common edge, and is discussed below. Note also that thereis symmetry to the routing of the lines on the module. In particular,the right angle turns are on both sides of the device locations. Thisallows signals to enter and leave the module from either end relative tothe orientation key. In either case, the signals still arrive at thesame time at the device locations if they started at the same time atthe non-terminated end.

In FIG. 8 it is allowed that bus segments 810 a-c can be segments on themotherboard if there are no modules. These segments 810 a-c aredistinguished from segments 800 by whether there are devices present onthe segment. Sections 800 have no devices connected between the ends ofthe segment whereas sections 810 a-c have devices connected between theends.

The routing of the clock segments is shown in FIG. 9. Again segments 900are routed on the motherboard and segments 900 a-c are those routed onthe module. In the case of the clock line, the clock originates fromclock source 980 typically located on the motherboard is routed throughthe modules to the loop 905, typically on the motherboard, back throughthe modules 900 a-c and terminates at the termination device 940, alsotypically on the motherboard. Also shown are the right angle turns 960that the clock lines must make on the modules 910 a-c. Keys 930 areshown to indicate that all modules are coupled to the motherboard withthe same orientation. Again, the so-called module segments 910 a-c neednot be physically separate from the motherboard, but instead may besegments on the motherboard having devices connected between the ends ofthe segments. However, no devices are connected between the ends ofsegments 900.

FIG. 10 shows an alternate embodiment of the routing of the data line.In the case of FIG. 10 the routing of the line terminates at atermination device 1040 on the module 1010 rather than the motherboard.Again the keys 1030 are shown to give the orientation. Thus, the modulehaving the termination device is the last module in the chain ofsegments. The clock segment of FIG. 9 could also have the terminationdevice on the last module in a manner similar to the segment of FIG. 10.

FIG. 11 shows the routing of a data line according to an embodiment ofthe present invention such that the route always enters the keyed end ofthe module 1110 requiring two folds 1100 on the motherboard. Thedifference between this embodiment and the one shown in FIGS. 8 and 9 isthat the routes in the latter embodiment are shorter, thus reducing thetime it takes a signal to propagate from one end to the other end of theline. In some cases it may be necessary to route the bus segments in thechain as shown in FIG. 11.

When connectors are used as the means of coupling between bus segments,certain characteristics of the connector are important with respect totheir effect on the transmission lines coupled by the connector. FIG. 12shows a circuit model of a connector and the transmission lines oneither side and near the connection made by the connector. Section 1240shows an equivalent circuit representation of the transmission linesegment on the motherboard near the connector. Section 1250 shows thesegment of transmission line on the module near the connector andsection 1260 shows a circuit model of the connector itself. In themotherboard section 1240, the unloaded impedance of the line 1200 is Z0,which is preferably in the range of 22 to 32 ohms and more preferably 28ohms. Motherboard section 1240 also shows the capacitance CMB bycapacitor 1220. This capacitance results from a through-hole, surfacemount pad or other capacitive structure, placed on the motherboard inthe path of the line 1270. The connector section 1260 includes thecapacitance CPIN of the connector pin shown as capacitor 1240 and theinductance LPIN of the pin shown as inductor 1230. Finally, section 1250includes a pad on the module having capacitance CPAD and shown ascapacitor 1255. Line 1210 in module section 1250 has the same preferredimpedance Z0 as the line 1200 in motherboard module 1240. By controllingthe value of CMB and CPAD, the equivalent impedance of the pin isaltered to become the preferred impedance Z0. It is preferred forconnectors used in connection with an embodiment of the presentinvention that the pin inductance be in the range of 2 to 3 nanohenrys(Nh). A typical value is approximately 2.3 Nh. The pin capacitance is inthe range of 0.5 to 1.0 pico-farads (Pf) and is typically about 0.6 Pf.If the value of CMB and CPAD are made to be about 1 Pf, the totalcapacitance near the connector is about 2.6 Pf and the total inductanceis about 2.3 nH. The result is an effective impedance of about 28-30ohms for the connector. Thus, a signal propagating from the motherboardsection 1240 through the connector section 1260 on onto the modulesection 1250 encounters no significant change of impedance and nosignificant reflection is generated. Furthermore, the time to passthrough the connector, i.e., from beginning of section 1240 to the endof section 1250 is uniform across all of the signals.

FIG. 13A shows a circuit model for the transmission line segment on amodule in an embodiment of the present invention, where the term“module” implies the presence of devices connected to the transmissionline segment. In this figure, connection points 1350 represent thecoupling between the motherboard and the module, which may be by meansof a connector whose characteristics were discussed above. Line sections1360 are those portions of the transmission line having impedance Z0 andno device connections, and line sections 1320 comprise sections oftransmission line 1355 having an unloaded line impedance of Z1 and adevice load 1340. FIG. 13B shows an equivalent circuit for the deviceload indicating that it can be modeled approximately as a capacitor CDL.The value of CDL is approximately in the range of 2-3 Pf, but preferablyabout 2.5 Pf for a single device connection. To avoid discontinuitiesbetween the Z0 section of line 1360 and the portion of line 1320 havingthe device load, the Z1 impedance of line sections 1355 is increasedsubstantially over the Z0 value so that the equivalent impedance of linesection 1320 matches that of the Z0 section 1360. If the nominalimpedance of the Z0 section 1360 is about 28 ohms and a device load isto a first order capacitive and approximately 2.5 pF, then Z1 is madeapproximately 70 ohms. The result is that the additional capacitancereduces the impedance of section 1320 to about 28 ohms according to thewell known relationship that the loaded impedance Z1′ equals theunloaded impedance Z1 multiplied by the square root of the ratio of theunloaded capacitance C1 to the total capacitance CT.Z1′=Z1sqrt(C1/CT) where CT=C1+CDL,where CDL is the device load.

Thus, adding capacitance to the line lowers its impedance. A constrainton the above calculation is that the length of section 1320 must be onthe order of an inch or less, preferably about 0.25 inches, implyingthat the device loads must be spaced by about 0.25 inches apart. If thedevice loads are spaced more widely, say by greater than 1 inch, then aline with an impedance of Z0 must be placed between the device loads sothat distance over which the device loading has an effect is reduced toless than an inch. The reason for the distance limitation is based thetransition time of the signal and how far that signal can travel duringits transition time. Preferably, the signal to be propagated over theline has a rise time of about 200 pico-seconds (pS). During that timethe signal will travel about an inch for a line on the surface of themodule, assuming that signals on the surface of the module have a flighttime of 150 to 200 pS/inch (to be discussed below). If there are nodevice loads on the module because no devices are present (in the caseof a continuity module) then line sections 1320 are not present. Onlyline sections 1360 having an impedance of Z0 are present and run thelength of the module.

FIG. 14A shows more detail regarding the routing of the lines on themodule 1400 and, in particular, shows the right angle turn taken by aline 1420 routed on the near surface of the module and a line 1430routed on the far surface of the module. The purpose of the right angleturn is to assure equal and shortest lengths of lines across the module.This helps to assure uniform arrival times of the signals at each of thedevices on the module. These lines 1420 and 1430 are received by vias1460 at which point the lines change direction by 90 degrees to runparallel with the length of the module on a pair of internal sections1440 a,b per line. Dual internal tracks are necessary to match theimpedance of the line on the surface 1320 to the impedance of the lineinternally 1440 a,b as will be discussed below. Thus, sections 1420 and1440 a,b comprise section 1360 in FIG. 13A Sections 1440 a,b continuetheir routes down the length of the module until reaching the sectionwhere the devices are located on the module. At this point the sectionscome back to the surface of the module by means of vias 1410 to connectto the device. Sections 1470 of the line comprise section 1320 in FIG.13A. Sections 1470 continue along the length of the module until a pointis reached where there are no more devices to connect to the line. Thelines then exit the board in a fashion similar to how the lines entered,again by dual tracks internally for additional routing along the length,then into vias to make the right angle turn and then running on eithersurface of the module to the connection point of the module with themotherboard. Continuity modules placed anywhere except after the lastmodule having devices on it, should have equal length lines to assurethat signals entering the module at the same time leave the module atthe same time. Therefore, if turns are used to route the lines of thecontinuity module, they should be right angle turns to maintain equallengths for the lines.

FIG. 14B shows a magnified view of the right angle routing. As can beseen from the figure the length of any line from point A to point B isalways equal to 10 units. Line 1420 is routed on one side of the moduleto a feed through hole 1460 at which point the line 1440 a continues ona different layer. Line 1430 is routed on the opposite side of themodule from line 1420 to a feed through hole 1460 at which point theline 1440 b continues on the same layer as 1440 a. This right anglerouting thus requires at least three signal layers for routing if lines1420 and 1430 must be routed on opposite sides of the module, possiblyto match the pitch of any connector to which they may connect. Only twolayers are required when lines 1420 and 1430 are routed from the sameside of the module.

FIGS. 15A and 15B give a perspective view of the sections of line andthe internal construction of the module. In FIG. 15A sections 1520correspond to sections 1420 in FIG. 14 and sections 1540 a,b correspondto sections 1440 a,b in FIG. 14. Sections 1510 in FIG. 15 correspond tosections 1470 in FIG. 14A and are the sections that run at the site ofthe devices on the module. FIG. 15A also shows the two reference planes1560 internal to the module to maintain a symmetric structure. Theseplanes are typically at ground potential and provide a return path forthe currents traveling on the lines. Vias 1550 connect the two referenceplanes 1560 together at various points near the via 1525 and internalsection 1575 which connects the lines 1520 to the dual internal sections1540 a,b.

FIG. 15B shows the two reference planes 1560, a section 1522 entering onthe opposite side of the module from section 1520 in FIG. 15A. Internalsections 1540 a,b are shown as in FIG. 15A between the two referenceplanes 1560. Section 1570 of the line in FIG. 15B results from theplating through of via 1525 in FIG. 15A. The plating is done so that thedual internal tracks 1540 a,b can be connected to section 1520 in FIG.15A. Filled via 1555 is the result of plating through via 1550 in FIG.15A which serves to connect the two reference planes together. Sincesection 1520 in FIG. 15A only connects on one side of the module butcouples to a pair of internal sections 1540 a,b, it is necessary toprovide for a return path for the image currents induced by the currentson the internal sections. The section 1565 in FIG. 15A connecting thetwo reference planes together provides the return path for the imagecurrents. The section 1565 connecting the two reference planes also hasthe purpose of reducing cross-talk between the signal lines. This effectresults from the placement of the section 1565 near the section 1575coupling the two internal sections to each other and to the signalsection 1520, causing the section 1565 to act as a partial shieldbetween the signal lines.

FIGS. 16A-16E show cross-section views of the internal construction ofthe module for various embodiments of the present invention. As in FIGS.15A and 15B, dual internal sections 1640 a and 1640 b, shown in View A,are shown between the two reference planes 1660. Section 1620 is a linesection running on the surface of the module away from the sites of thedevices and corresponds to section 1520 in FIG. 15A. Section 1675, inViews A and B, is the section that connects the dual internal sections1640 a and 1640 b together. Section 1625 is the section of the line thatruns on the surface of the module near the location of the IC devices,leaving the space between the two reference planes open for additionallines or reference planes. The additional planes or lines 1670 a,b, areshown in View D of FIG. 16. In View D the additional planes may be aplane which provides power to the devices (a power plane) and areference voltage (Vref) plane for use by all of the devices. Theadditional planes are placed close to the reference plane to improve thequality of coupling to the reference planes. This helps reduce noise onthe additional planes. In another embodiment, shown in View C, section1627, runs internally rather than on the surface near the location ofthe IC devices. View E of FIG. 16 shows an alterative embodiment forrouting lines internally. Rather than route the lines as in View A wherethe two lines 1640 a and 1640 b are connected to the same signal 1620 byfeedthrough 1675 and run in parallel between the same reference planes1660, View E shows a case where the two lines 1640 a and 1640 b are runin parallel but have their own reference planes. In particular, line1642 a has 1661 and 1662 as its reference planes, and 1642 b has 1664and 1665 as its reference planes. In View E, segments of line having ICdevices connected between the ends have one of the lines, either 1640 aor 1640 b, eliminated. In segments of line having no IC devicesconnected between the ends, the lines run in parallel connected to thesame signal.

Because sections of the line run on the surface of the module and somesections run internally between the two reference planes, the sectionsof line have different characteristics such as impedance and time oftravel (flight time). Sections of line, such as either 1640 a or 1640 bindividually, have a different impedance than section 1620 or 1625running on the surface of the module. These two types of lines are shownin FIGS. 17 and 18. It is important to match the impedance of the twotypes of lines and to compensate for any difference in flight time toachieve uniform arrival times of signals at the devices.

FIG. 17 shows, in cross-section, the important parameters for thesection of line running on the surface of the module away from thepresence of devices, i.e., section 1620 in FIG. 16A. In FIG. 17, theline 1700 has width, W, thickness, t, and runs over a dielectricmaterial 1710 having thickness, S, and permittivity, ε_(r). Ground plane1720 is located opposite the line 1700 on the other side of thedielectric. In this construction, commonly referred to as a microstrip,the impedance of the line is determined to a first order by the ratioW/S and the permittivity ε_(r) of the dielectric. (See Matick,Transmission Lines For Digital and Communication Networks, IEEE Press,1995, p. 326.). It is preferred to have the width of the line, W, in therange of 20-24 mils, preferably 22 mils to match the pitch of anyconnector coupled to the line. To construct a line having an impedanceof approximately 25-30 ohms and preferably 28 ohms with a materialhaving a permittivity of 4.5, the separation of the line S from theground plane is in the range of 4.4 to 5.3 mils, preferable 5 mils,giving a W/S ratio of about 4.5. A smaller width line W or a greaterseparation distance S results in an impedance that is greater than thepreferred range of 25 to 30 ohms. Thus, segments of line traveling onthe surface of the module and without devices connected between the endshave an impedance in the range of 25 to 30 ohms.

To construct a line of the type shown in FIG. 17, but having animpedance of approximately 70 ohms with the same dielectric material forlines on the surface of the module and to which the devices connect, theW/S ratio is altered to become approximately equal 1.0. Thus, byaltering the width of the trace from approximately 22 mils to 5 mils,and holding the distance S constant, a line running on the surface ofthe module near the IC devices, section 1625 in FIG. 16B, is constructedhaving an impedance of about 70 ohms. As discussed above, when this lineis taken in combination with the device load, the impedance of thesection is reduced to about 28 ohms, thus matching the impedance of anline section which is does not have any devices connected to it.

FIG. 18 shows, in cross-section, the important parameters for a sectionof line having no devices connected to it and running internally betweenthe two reference planes of the module, i.e., section 1640 a or 1640 bin FIG. 16A. In FIG. 18, the width of the line 1800 is W and itsthickness is t. The separation distance between the two reference planes1820 is S′ and the dielectric filling the separation distance has apermittivity of ε_(r). In this construction, the impedance of the lineis determined to a first order by the W/S′ ratio, the t/S′ ratio and thepermittivity ε_(r). (See Matick, Transmission Lines For Digital andCommunication Networks, IEEE Press, 1995, p. 327.) The W/S′ ratio willtypically be less than 1 for lines having practical values of tracewidths, W, of about 22 mils or less, determined by connector pitch and apractical separation distance S′ between planes of approximately 35 milsdetermined by module thickness required for insertion of the module intoa socket. Under these circumstances, the impedance of a line shown inFIG. 18, commonly referred to as a stripline, will be approximately 50ohms when W is approximately 14 mils, S′ is approximately 35 mils and tis approximately 1.4 mils (for 1 ounce copper plating). This impedancedoes not match the preferred impedance of the other lines to which theinternal lines connect assuming the use of the same dielectric material.Therefore, two lines, connected to the same signal, are run in parallelinternally as is shown in FIGS. 16 and 19. The use of two lines reducesthe impedance from 50 ohms to about 28 ohms (rather than 25 ohms) due toinductive coupling that is present between the lines. Thus, the use ofthe dual parallel lines creates a internal section of line that matchesthe impedance of the other lines to which the internal section connects.

To construct a line of the type in FIG. 18 having an impedance ofapproximately 70 ohms for sections having devices connected, such assection 1627 in View C of FIG. 16, the W/S′ ratio is altered to about0.15 by decreasing the width of the trace from about 14 mils to 5.5mils. The thickness of the trace, t, and the other parameters are heldconstant. To construct a line of the type in FIG. 18 having an impedanceof approximately 50 ohms, such as for section 1642 a or 1642 b in View Eof FIG. 16, the separation distance S′ is about 15 mils, the line widthW is about 3 mils, and the line thickness is 1.4 mils. This creates aW/S′ ratio of about 0.2 and a t/S′ ratio of about 0.1. The impedancethen is about 56 ohms for ε_(r) equal to 4.5. Thus, two lines 1642 a and1642 b are run in parallel and connected to the same signal to give animpedance of 28 ohms for segments of the line without IC devices. Nearthe IC devices one of the lines 1642 a or 1642 b is eliminated resultingin a segment of line having an impedance of 56 ohms for segments of theline with IC devices. With device loads taken into account, theimpedance is reduced to 28 ohms.

There is still a problem associated with the use of the two types oflines as shown in FIGS. 17 and 18, even if the lines are constructed sothat the impedance is the same for each type. The problem is that thetime for a signal to travel a unit distance (flight time) for a lineconstructed as in FIG. 17 is faster than the flight time for a lineconstructed as in FIG. 18. The expression for the flight time TF isTF=TCsqrt(ε′_(r)),where TC is the flight time of a signal in free space (approximately84.7 picoseconds per inch=3.33 pS/millimeter) and ε′_(r) is theeffective relative permittivity of the medium in which the signalpropagates. For lines constructed according to FIG. 18 or 19, the flighttime is about 180 pS/in (picoseconds per inch), because the effectiverelative permittivity is about 4.5. The flight time of a lineconstructed according to FIG. 17 is more difficult to determine becausethe effective relative permittivity has a contribution from both thedielectric 1710 and air surrounding the top and sides of the conductor1700. The effective relative permittivity is less than that of thedielectric and in the range of 3.0 to 3.15. Thus, according to theequation, flight time for such a line is in the range of about 145 pS/into 150 pS/in. Compared to the line in FIG. 18, the line in FIG. 17 isabout 17% to 20% faster.

To assure that signals traveling on the lines of the module arrive atthe IC device at the same time requires some form of compensation whichtakes into account the different flight times of the different types oflines. In one embodiment of the present invention, the physical line isfirst made to have the same length by routing the lines with a rightangle turn as shown on FIG. 14A,B. Right angle turning or “folding” ofthe line assures that the length of section 1420 added to the length ofsection 1440 a is the same for all of the folded lines. This helps tocut down on the differences between arrival times at the IC devicelocations 1410 in FIG. 14A,B. Next, the slowest path to a device isidentified. In the embodiment shown in FIG. 14A,B, the slowest path isthe path comprising sections 1420 and 1440 a. This path is slowestbecause the signals on this path travel the longest distance over asection of the line with the slowest flight time. Finally, delaymatching segments, 1480 in FIG. 14 (and 530 in FIG. 5) are added to thefaster paths so that the signals on them arrive at the IC device site1410 at the same time as the slowest path. Thus, folding the group oflines, assures that the lines have equal lengths except for the delaycompensating sections which are then added to compensate for the unequalflight times between the two types of lines. After compensation, signalsentering the module take the same time to travel to the site of the ICdevices 1410 regardless of the particular routing path of the line.

Continuity modules used to couple the bus to modules having devices onthem, should have delay matching segments added if the routes on thecontinuity module use right angle turns with portions of the turn havingdifferent flight times. The delay matching segments assure that signalsentering the continuity module at the same time, leave the continuitymodule at the same time. A continuity module connected to thetermination device need not have any delay matching segments added asuniform arrival times of signals at the termination device is notnecessary.

Although the invention has been described in considerable detail withreference to certain embodiments thereof, other embodiments arepossible. For example, FIG. 7 shows another embodiment of the presentinvention in which the devices are mounted on the motherboard directly.The bus has sections 790 in which no devices are present and a section795 with devices connected. The data lines are routed from thecontroller or master 700 to the individual devices 755 and then to theterminator device 740. Near the devices, the width of the data lines isaltered to maintain an impedance that matches the nominal impedance ofthe sections 790 where there are no devices 755. Where data lines mustturn in their route from the controller to the devices or from thedevices to the terminating device, the turns are right angle turns tomaintain the same length of each line. In addition, delay matchingsegments are added where there are sections of the route that havedifferent flight times. Thus the arrival time of signals to the devicesin the device segment of the line 795 is still assured.

In FIG. 7, the clock lines are routed from the clock source 780 to thedevices 755 to the controller 700. At or near the controller the clockline reverses direction by means of clock loop 725 and continues on line770 to a terminator device 745. The clock lines have uniform impedancethroughout the length of the line, again by altering the width of theclock sections near the devices to match the impedance of sections 790without devices. Again, right angle turns are employed to maintain thesame length of line as the data lines. Delay matching segments are addedwhere there are sections of the route that have different flight times.Another embodiment that is possible is locating the controller 400depicted in FIG. 4 on a module 420 rather than on the motherboard 410.Therefore, the spirit and scope of the appended claims should not belimited to the description of the preferred versions contained therein.

1. A module comprising: a substrate; a plurality of signal linesincluding signal lines to carry data interleaved with signal lines thatconvey a ground potential voltage; the plurality of signal linesincluding first and second signal lines routed alongside one another,each of the first and second signal lines comprising a first segmentconnected to a second segment via a turn portion such that the firstsegment and the second segment are oriented in different directions, andwherein for each of the first and second signal lines, a respectivesignal, starting at a corresponding first edge finger disposed on thesubstrate, traverses in sequence, a respective first segment of arespective signal line, a respective turn portion of the respectivesignal line, and a respective second segment of the respective signalline; a clock line that includes a first segment connected to a secondsegment via a turn portion such that the first and second segments ofthe clock line are oriented in different directions, the clock line toprovide a clock signal that traverses in sequence, a second edge finger,the first segment of the clock line, the turn portion of the clock line,and the second segment of the clock line; and a plurality of memorydevices disposed on the substrate, wherein each memory device of theplurality of memory devices is connected to the plurality of signallines, including the second segment of the first and second signal linesand the second segment of the clock line such that the respectivesignals traversing the first and second signal lines and the clocksignal traversing the clock line arrive at the plurality of memorydevices in sequence.
 2. The module of claim 1, further comprising aplurality of first termination elements and second termination element,wherein: for each of the first and second signal lines, after traversingthe respective first segment of a respective signal line, the respectiveturn portion of the respective signal line, and the respective secondsegment of the respective signal line, the respective signal terminatesat a respective first termination element; and after traversing thefirst segment of the clock line, the turn portion of the clock line, andthe second segment of the clock line, the clock signal terminates at thesecond termination element.
 3. The module of claim 2, wherein respectivelengths of the first and second signal lines between the respectivefirst edge finger and the respective first termination element aresubstantially equal to a length of the clock line between the secondedge finger and the second termination element.
 4. The module of claim1, wherein each of the turn portions of the signal lines are right angleturn portions, such that the first segments of the first and secondsignal lines are oriented perpendicular to the second segments of thefirst and second signal lines.
 5. The module of claim 1, wherein each ofthe turn portions of the signal lines comprise a via that is routedthrough respective feedthrough holes in the substrate of the module. 6.The module of claim 1, wherein the substrate includes a first side and asecond side, wherein the first side is positioned opposite to the secondside, and wherein a first memory device of the plurality of memorydevices is disposed on the first side of the substrate and a secondmemory device of the plurality of memory devices is disposed on thesecond side of the substrate.
 7. The module of claim 1, wherein themodule includes at least three signal layers, and wherein: each firstsegment of the first and second signal lines is routed on the first sideof the substrate, each of the turn portions of the signal linescomprises a via that is routed through a respective first feedthroughhole in the substrate, and each second segment of the first and secondsignal lines is routed in an internal layer of the substrate; and thefirst segment of the clock line is routed on the second side of thesubstrate, the turn portion of the clock line comprises a via that isrouted through a second feedthrough hole in the substrate, and thesecond segment of the clock line is routed in the internal layer of thesubstrate.
 8. The module of claim 1, wherein the respective firstsegments of the first and second signal lines are disposed along a widthof the substrate and the respective second segments of the first andsecond signal lines are disposed along a length of the substrate suchthat the length is oriented perpendicular to the width.
 9. The module ofclaim 1, wherein the respective first segments of the first and secondsignal lines are routed substantially parallel to a first edge of themodule and the respective second segments of the first and second signallines are routed substantially parallel to a second edge of the module,wherein the second edge is longer than the first edge.
 10. A modulecomprising: a substrate; a plurality of edge fingers, including firstand second edge fingers, disposed at a first edge of the substrate, theplurality of edge fingers including edge fingers to receive datainterleaved with edge fingers to receive a ground potential voltage; asignal line that includes a first segment, first turn portion, secondsegment, second turn portion, and third segment such that a signalstarting at the first edge finger traverses in sequence the firstsegment of the signal line, the first turn portion of the signal line,the second segment of the signal line, the second turn portion of thesignal line, and the third segment of the signal line; a clock line thatincludes a first segment, first turn portion, second segment, secondturn portion and third segment, such that a clock signal starting at thesecond edge finger traverses in sequence the first segment of the clockline, the first turn portion of the clock line, the second segment ofthe clock line, the second turn portion of the clock line, and the thirdsegment of the clock line; and a plurality of memory devices disposed onthe module, wherein each memory device of the plurality of memorydevices is connected to the second segment of the signal line and secondsegment of the clock line such that the signal traversing along thesignal line and the clock signal traversing along the clock line arriveat each memory device of the plurality of memory devices in sequence.11. The module of claim 10, further including a plurality of signallines routed alongside the signal line such that each signal line of theplurality of signal lines includes, respectively, a first segment thatis routed alongside the first segment of every other signal line of theplurality of signal lines, a second segment that is routed alongside thesecond segment of every other signal line of the plurality of signallines, then traverses a third segment that is routed alongside the thirdsegment every other signal line of the plurality of signal lines. 12.The module of claim 10, wherein the first turn portion of the signalline is a right angle turn portion, such that the first segment of thesignal line is oriented perpendicular to the second segment of thesignal line.
 13. The module of claim 10, wherein each of the first andsecond turn portions of the signal line are right angle turn portions,such that the first segment of the signal line is oriented perpendicularto the second segment of the signal line and the second segment isoriented perpendicular to the third segment of the signal line.
 14. Themodule of claim 10, wherein each of the first and second turn portionsof the signal line comprises a respective via that is routed through arespective feedthrough hole in the substrate of the module.
 15. Themodule of claim 10, wherein a first memory device of the plurality ofmemory devices is disposed on a first side of the substrate and a secondmemory device of the plurality of memory devices is disposed on a secondside of the substrate, wherein the first side is oriented opposite thesecond side.
 16. The module of claim 10, wherein the module includes atleast three signal layers, and wherein: the first segment of the signalline is routed on a first side of the substrate, the first turn portionof the signal line comprises a via that is routed through a firstfeedthrough hole in the substrate, and the second segment of the firstsignal line is routed in an internal layer of the substrate, and thefirst segment of the clock line is routed on the second side of thesubstrate, the first turn portion of the clock line comprises a via thatis routed through a second feedthrough hole in the substrate of themodule, and the second segment of the clock line is routed on theinternal layer of the substrate.
 17. The module of claim 10, wherein thefirst segment and third segment of the signal line are disposed along awidth of the substrate and the second segment of the signal line isdisposed along a length of the substrate such that the length isoriented perpendicular to the width.
 18. The module of claim 10, whereinthe first segment of the signal line is routed substantially parallel toa first edge of the module and the second segment of the signal line isrouted substantially parallel to a second edge of the module, whereinthe second edge is longer than the first edge.
 19. An apparatuscomprising: a substrate; a controller device disposed on the substrate;a plurality of signal lines coupled to the controller device, theplurality of signal lines including signal lines to carry datainterleaved with signal lines that convey a ground potential voltage;the plurality of signal lines including a signal line having a firstsegment connected to a second segment via a turn portion, such that thefirst segment and the second segment are oriented in differentdirections, such that a signal starting at the controller device,traverses in sequence the first segment of the signal line, the turnportion of the signal line, and the second segment of the signal line; aclock line that includes a first segment connected to a second segmentvia a turn portion such that the first and second segments of the clockline are oriented in different directions, such that a clock signalstarting at the controller device, traverses in sequence a first segmentof the clock line, a turn portion of the clock line, and a secondsegment of the clock line; and a plurality of memory devices, whereineach memory device of the plurality of memory devices is connected tothe second segment of the signal line and second segment of the clockline such that the signal traversing along the signal line and the clocksignal traversing along the clock line arrive at each memory device ofthe plurality of memory devices in sequence.
 20. The apparatus of claim19, further comprising a plurality of first termination elements and asecond termination element, wherein: after traversing the first segmentof the signal line, the turn portion of the signal line, and the secondsegment of the signal line, the respective signal terminates at a firsttermination element; and after traversing the first segment of the clockline, the turn portion of the clock line, and the second segment of theclock line, the clock signal terminates at the second terminationelement.
 21. The apparatus of claim 19, wherein the substrate is amotherboard comprising a socket, and wherein the plurality of memorydevices are disposed on a module having a connector interface that iscapable of being inserted into the socket.
 22. The apparatus of claim21, wherein the first segment of the signal line, turn portion of thesignal line, and second segment of the signal line are routed on thememory module.
 23. The apparatus of claim 19, wherein the turn portionof the signal line is a right angle turn portion, such that the firstsegment of the signal line is oriented perpendicular to the secondsegment of the signal line.
 24. The apparatus of claim 19, wherein theplurality of signal lines includes a groups of signal lines routedalongside the signal line such that each signal line of the group ofsignal lines includes, a respective first segment, a respective turnportion, and a respective second segment that are routed alongside therespective first segment of the signal line of the signal line, turnportion of the signal line, and second segment of the signal line.